Stereo rendering system

ABSTRACT

A method includes receiving an indication of a field of view associated with a three-dimensional (3D) image being displayed on a head mount display (HMD), receiving an indication of a depth of view associated with the 3D image being displayed on the HMD, selecting a first right eye image and a second right eye image based on the field of view, combining the first right eye image and the second right eye image based on the depth of view, selecting a first left eye image and a second left eye image based on the field of view, and combining the first left eye image and the second left eye image based on the depth of view.

FIELD

Embodiments relate to rendering left eye and right eye images and/orvideo of a stereo image and/or video.

BACKGROUND

Typical stereo rendering involves computing a dense optical flow fieldbetween pairs of cameras, and then interpolating a viewpoint over theentire 3D image. This is difficult and even might be consideredimpossible in some cases, such as semi-transparent objects. Even fornormal solid objects, this is difficult because most optical flowalgorithms are too slow to be done in real-time. In other words,interpolating 3D images from captured 2D images can be processorintensive. As a result, generating 3D images and/or 3D video in realtime to accomplish a desired playback user experience can be difficult.Therefore, it is desirable to render 3D images and/or 3D video withoutoptical flow interpolation in real time and/or as the image or video isstreamed.

SUMMARY

Example embodiments describe systems and methods to 3D images and/orvideo.

In a general aspect, a method includes receiving an indication of afield of view associated with a three-dimensional (3D) image beingdisplayed on a head mount display (HMD), receiving an indication of adepth of view associated with the 3D image being displayed on the HMD,selecting a first right eye image and a second right eye image based onthe field of view, combining the first right eye image and the secondright eye image based on the depth of view, selecting a first left eyeimage and a second left eye image based on the field of view, andcombining the first left eye image and the second left eye image basedon the depth of view.

In another general aspect, a method includes streaming athree-dimensional (3D) video to a head mount display (HMD). Each frameof the 3D video includes a left eye image and a right eye image. Themethod further includes generating a subsequent frame which includesdetermining a field of view associated with the 3D video, determining adepth of view associated with the 3D video, selecting a first right eyeimage and a second right eye image based on the field of view, combiningthe first right eye image and the second right eye image as the righteye image based on the depth of view, selecting a first left eye imageand a second left eye image based on the field of view, and combiningthe first left eye image and the second left eye image as the left eyeimage based on the depth of view.

In yet another general aspect, a non-transitory computer-readablestorage medium having stored thereon computer executable program codewhich, when executed on a computer system, causes the computer system toperform steps. The steps include receiving an indication of a field ofview associated with a three-dimensional (3D) image being displayed on ahead mount display (HMD), receiving an indication of a depth of viewassociated with the 3D image being displayed on the HMD, selecting afirst right eye image and a second right eye image based on the field ofview, combining the first right eye image and the second right eye imagebased on the depth of view, selecting a first left eye image and asecond left eye image based on the field of view, and combining thefirst left eye image and the second left eye image based on the depth ofview.

Implementations can include one or more of the following features. Forexample, selecting of the first right eye image and of the second righteye image can include determining a right eye position of a user of theHMD, selecting the first right eye image as an image taken by a firstcamera positioned to the right of the right eye position, and selectingthe second right eye image as an image taken by a second camerapositioned to the left of the right eye position.

For example, selecting of the first left eye image and of the secondleft eye image can include determining a left eye position of a user ofthe HMD, selecting the first left eye image as an image taken by a firstcamera positioned to the right of the left eye position, and selectingthe second left eye image as an image taken by a second camerapositioned to the left of the left eye position. The first right eyeimage, the second right eye image, the first left eye image and thesecond left eye image can be selected from a plurality of imagescaptured at substantially the same moment in time. The combining of thefirst right eye image and the second right eye image can includeshifting the first right eye image with respect to the second right eyeimage until a portion of a combined image, based on the depth of view,is sharp. The combining of the first right eye image and the secondright eye image can include shifting both the first right eye image andthe second right eye image toward the center of the field of view untila portion of a combined image, based on the depth of view, is sharp.

For example, the combining of the first right eye image and the secondright eye image can include color merging such that a portion of thecombined image has substantially a same color palette as a correspondingportion of at least one of the first right eye image and the secondright eye image. The combining of the first right eye image and thesecond right eye image can include color merging using a color offsetbased on a weighted offset associated with a camera distance from thecenter of the field of view. The first left eye image, the second lefteye image, the first left eye image and the second left eye image can beselected from a plurality of images captured at substantially the samemoment in time. The combining of the first left eye image and the secondleft eye image can include shifting the first left eye image withrespect to the second left eye image until a portion of a combinedimage, based on the depth of view, is sharp.

For example, the combining of the first left eye image and the secondleft eye image can include shifting both the first left eye image andthe second left eye image toward the center of the field of view until aportion of a combined image, based on the depth of view, is sharp. Thecombining of the first left eye image and the second left eye image caninclude color merging the combined image such that a portion of thecombined image has substantially a same color palette as a correspondingportion of at least one of the first left eye image and the second lefteye image. The combining of the first left eye image and the second lefteye image can include color merging the combined image using a coloroffset based on a weighted offset associated with a camera distance fromthe center of the field of view.

For example, selecting of the first right eye image and of the secondright eye image can include determining a right eye position of a userof the HMD, selecting the first right eye image as an image taken by afirst camera positioned to the right of the right eye position, andselecting the second right eye image as an image taken by a secondcamera positioned to the left of the right eye position. For example,selecting of the first left eye image and of the second left eye imagecan include determining a left eye position of a user of the HMD,selecting the first left eye image as an image taken by a first camerapositioned to the right of the left eye position, and selecting thesecond left eye image as an image taken by a second camera positioned tothe left of the left eye position.

BRIEF DESCRIPTION OF THE DRAWINGS

Example embodiments will become more fully understood from the detaileddescription given herein below and the accompanying drawings, whereinlike elements are represented by like reference numerals, which aregiven by way of illustration only and thus are not limiting of theexample embodiments and wherein:

FIG. 1 illustrates a block diagram of a system for capturing andrendering an image and/or video according to at least one exampleembodiment.

FIG. 2 illustrates a block diagram of an image processing systemaccording to at least one example embodiment.

FIG. 3A illustrates a top view perspective of a block diagram of animage capture configuration according to at least one exampleembodiment.

FIG. 3B illustrates a front view perspective of a block diagram of theimage capture configuration according to at least one exampleembodiment.

FIG. 4 illustrates a block diagram of a method according to at least oneexample embodiment.

FIGS. 5A, 5B and 5C illustrate an image as captured by cameraspositioned to the center, to the left and to the right of a scene to becaptured according to at least one example embodiment.

FIGS. 6A and 6B illustrate the same image as viewed by eyes along thesame plane as the cameras according to at least one example embodiment.

FIGS. 7A, 7B and 7C illustrate the image as visualized by the humanvisual system.

FIGS. 8A and 8B illustrate diagrams of combined images according to atleast one example embodiment.

FIG. 8C illustrates a visualized image based on the combined images whendisplayed on a display of a HMD according to at least one exampleembodiment.

FIGS. 9A and 9B illustrate diagrams of combined images according to atleast one example embodiment.

FIG. 9C illustrates a visualized image when displayed on a display of aHMD according to at least one example embodiment.

FIGS. 10A and 10B illustrate diagrams of combined images according to atleast one example embodiment.

FIG. 10C illustrates a visualized image when displayed on a display of aHMD according to at least one example embodiment.

FIGS. 11A and 11B illustrate diagrams of a side by side comparison ofvisualized images at a far depth of view according to at least oneexample embodiment.

FIGS. 11C and 11D illustrate diagrams of a side by side comparison ofvisualized images at a mid depth of view according to at least oneexample embodiment.

FIGS. 11E and 11F illustrate diagrams of a side by side comparison ofvisualized images at a near depth of view according to at least oneexample embodiment.

FIGS. 12A and 12B illustrate block diagrams of a head mounted display(HMD) according to at least one example embodiment.

FIG. 13A illustrates a schematic representation of visual fields.

FIG. 13B illustrates the frontal plane and the midsagittal frontalplane.

FIG. 13C illustrates a plane orthogonal to the frontal planes thatbisects the eyes. Also shown are the gaze vectors from the eyes to apoint A.

FIG. 14 illustrates an example of a computer device and a mobilecomputer device.

It should be noted that these figures are intended to illustrate thegeneral characteristics of methods, structure and/or materials utilizedin certain example embodiments and to supplement the written descriptionprovided below. These drawings are not, however, to scale and may notprecisely reflect the precise structural or performance characteristicsof any given embodiment, and should not be interpreted as defining orlimiting the range of values or properties encompassed by exampleembodiments. For example, the relative thicknesses and positioning ofstructural elements may be reduced or exaggerated for clarity. The useof similar or identical reference numbers in the various drawings isintended to indicate the presence of a similar or identical element orfeature.

DETAILED DESCRIPTION OF THE EMBODIMENTS

While example embodiments may include various modifications andalternative forms, embodiments thereof are shown by way of example inthe drawings and will herein be described in detail. It should beunderstood, however, that there is no intent to limit exampleembodiments to the particular forms disclosed, but on the contrary,example embodiments are to cover all modifications, equivalents, andalternatives falling within the scope of the claims. Like numbers referto like elements throughout the description of the figures.

FIG. 1 is a block diagram of an example system 100 for capturing andrendering images and/or video in a 3D virtual reality (VR) environment.In the example system 100, a camera rig 102 can capture and provideimages over a network 104, or alternatively, can provide the imagesdirectly to an image processing system 106 for analysis and processing.In some implementations of system 100, the image processing system 106can store, modify and/or stream images and/or video based on imagescaptured by the camera rig 102. The image processing system 106 isdescribed in more detail below. In some implementations of system 100, amobile device 108 can function as the camera rig 102 to provide imagesthroughout network 104. Once the images are captured, the imageprocessing system 106 can perform a number of calculations and processeson the images and provide the processed images to a head mounted display(HMD) device 110 for rendering via network 104, for example. In someimplementations, the image processing system 106 can also provide theprocessed images to mobile device 108 and/or to computing device 112 forrendering, storage, or further processing.

The HMD device 110 may represent a virtual reality headset, glasses,eyepiece, or other wearable device capable of displaying virtual realitycontent. In operation, the HMD device 110 can execute a VR application(not shown) which can playback received and/or processed images to auser. In some implementations, the VR application can be hosted by oneor more of the devices 106, 108, or 112, shown in FIG. 1. In oneexample, the HMD device 110 can provide a video playback of a scenecaptured by camera rig 102. In another example, the HMD device 110 canprovide playback of still images stitched into a single scene.

The camera rig 102 can be configured for use as a camera (also can bereferred to as a capture device) and/or processing device to gatherimage data for rendering content in a VR environment. For example, indescribing the functionality of system 100, FIG. 1 shows the camera rig102 without cameras disposed around the rig to capture images. Otherimplementations of camera rig 102 can include any number of cameras thatcan be disposed around the circumference of a circular camera rig, suchas rig 102.

As shown in FIG. 1, the camera rig 102 includes a number of cameras 130and a communication system 132. The cameras 130 can include a singlestill camera or single video camera. In some implementations, thecameras 130 can include multiple still cameras or multiple video camerasdisposed (e.g., seated) side-by-side along the outer periphery (e.g.,ring) of the camera rig 102. The cameras 130 may be a video camera(s),an image sensor(s), a stereoscopic camera(s), an infrared camera(s), amobile device and/or the like. The communication system 132 can be usedto upload and download images, instructions, and/or other camera relatedcontent. The communication may be wired or wireless and can interfaceover a private or public network.

In some implementations, the camera rig 102 includes multiple digitalvideo cameras that are disposed in a side-to-side or back-to-backfashion such that their lenses each point in a radially outwarddirection to view a different portion of the surrounding scene orenvironment. In some implementations, the multiple digital video camerasare disposed in a tangential configuration with a viewing directiontangent to the circular camera rig 102. For example, the camera rig 102can include multiple digital video cameras that are disposed such thattheir lenses each point in a radially outward direction while beingarranged tangentially to a base of the rig. The digital video camerascan be pointed to capture content in different directions to viewdifferent angled portions of the surrounding scene.

In some implementations, the cameras can be configured (e.g., set up) tofunction synchronously to capture video from the cameras on the camerarig at a specific point in time. In some implementations, the camerascan be configured to function synchronously to capture particularportions of video from one or more of the cameras over a time period.Another example of calibrating the camera rig can include configuringhow incoming images are stored. For example, incoming images can bestored as individual frames or video (e.g., .avi files, .mpg files) andsuch stored images can be uploaded to the Internet, another server ordevice, or stored locally with each camera on the camera rig 102. Insome implementations, incoming images can be stored as encoded video.

In the example system 100, the devices 106, 108, and 112 may be a laptopcomputer, a desktop computer, a mobile computing device, or a gamingconsole. In some implementations, the devices 106, 108, and 112 can be amobile computing device that can be disposed (e.g., placed/located)within the HMD device 110. The mobile computing device can include adisplay device that can be used as the screen for the HMD device 110,for example. Devices 106, 108, and 112 can include hardware and/orsoftware for executing a VR application. In addition, devices 106, 108,and 112 can include hardware and/or software that can recognize,monitor, and track 3D movement of the HMD device 110, when these devicesare placed in front of or held within a range of positions relative tothe HMD device 110. In some implementations, devices 106, 108, and 112can provide additional content to HMD device 110 over network 104. Insome implementations, devices 102, 106, 108, 110, and 112 can beconnected to/interfaced with one or more of each other either paired orconnected through network 104. The connection can be wired or wireless.The network 104 can be a public communications network or a privatecommunications network.

In a HMD (e.g., HMD device 110), a viewer experiences a visual virtualreality through the use of a left (e.g., left eye) display and a right(e.g., right eye) display that projects a perceived three-dimensional(3D) video or image. According to example embodiments, a stereo or 3Dvideo or image is stored on a server (e.g., as a plurality of associated2D images as captured by the camera rig 102). The video or image can beencoded and streamed to the HMD from the server. The 3D video or imagecan be encoded as a left image and a right image which packaged (e.g.,in a data packet) together with metadata about the left image and theright image. The left image and the right image are then decoded anddisplayed by the left (e.g., left eye) display and the right (e.g.,right eye) display. According to example embodiments, the left image andthe right image can be images generated (e.g., combined as one image)based on two or more images synchronously captured. According to exampleembodiments, the left image and the right image can be images generated(e.g., combined as one image) based on a field of view and or a depth ofview

The system(s) and method(s) described herein are applicable to both theleft image and the right image and are referred to throughout thisdisclosure as an image, frame, a portion of an image, a portion of aframe, a tile and/or the like depending on the use case. In other words,the encoded data that is communicated from a server (e.g., streamingserver) to a user device (e.g., a HMD) and then decoded for display canbe a left image and/or a right image associated with a 3D video orimage.

The system 100 may include electronic storage. The electronic storagecan include non-transitory storage media that electronically storesinformation. The electronic storage may be configured to store capturedimages, obtained images, pre-processed images, post-processed images,etc. Images captured with any of the disclosed camera rigs can beprocessed and stored as one or more streams of video, or stored asindividual frames. In some implementations, storage can occur duringcapture and rendering can occur directly after portions of capture toenable faster access to content earlier than if capture and processingwere concurrent.

FIG. 2 illustrates a block diagram of an apparatus 200 including theimage processing system 106 according to at least one exampleembodiment. As shown in FIG. 2, the apparatus 200 can include at leastone memory 205 and at least one processor 210. The at least one memory205 can include the image processing system 106. The image processingsystem 106 can include at least one image/video source repository 215,an image selection module 220, a depth processing module 225 and animage combining module 230. The apparatus 200 can be an element of atleast one of the devices 106, 108, 110 or 112. Further, elements of theapparatus 200 can be divided amongst at least one of the devices 106,108, 110 or 112. For example, the at least one image/video sourcerepository 215 could be included in aforementioned electronic storageassociated with the image processing system 106 whereas the depthprocessing module 225 could be a subsystem of the HMD 110.

The at least one image/video source repository 215 can be configured tostore a plurality of 2D and/or 3D images and/or video. The images and/orvideo can represent scenes captured by camera(s) 130 and communicated tothe at least one image/video source repository 215 by the camera rig 102using the communication module 132. The images and/or video can havecorresponding metadata indicating camera position, field of view, imageresolution, capture time, frame order and/or the like.

The image selection module 220 can be configured to select 2D imagesfrom the image/video source repository 215 based on an indication of afield of view (e.g., received from the HMD 110). For example, the imageselection module 220 can select two or more right eye images and two ormore left eye images based on the field of view from the image/videosource repository 215. The two or more right eye images and the two ormore left eye images can be images of a scene taken by different cameras130 at substantially the same moment in time. The selection can be basedon the corresponding metadata for the images stored in the image/videosource repository 215.

The depth processing module 225 can be configured to determine a depthof view or focal depth of a user viewing an image and/or video using theHMD 110. For example, the HMD 110 can include an eye tracking device(e.g., first eye tracking sensor 1220-L and second eye tracking sensor1220-R illustrated in FIG. 12B) configured to determine the depth ofview and/or perform a measurement from which the depth of view can bedetermined For example, the eye tracking device can measure a positionof the pupils of the user's eyes. The eye tracking device can measureconvergence of the pupils and determine angle C (see FIG. 13A). Angle Ccan be used to determine the depth of view.

The image combining module 230 can be configured to combine two or moreimages in order to generate third image which is then communicated tothe HMD 110 for rendering on a display of the HMD 110. For example, theleft eye image can be generated by combining the two or more left eyeimages such that the combined image is sharp (e.g., not distorted or notblurred) at the depth of view. In order to generate a combined image afirst image (e.g., as captured by camera 320-1) can be shifted withrespect to a second image (e.g., as captured by camera 320-2) capturedat the same (or substantially the same) time as the first image until aportion of the combined image is sharp (e.g., the portion of the firstimage at the depth of view substantially overlaps the portion of thesecond image at the same depth of view). In an example implementation,the first image is shifted a number of pixels based on a distancebetween the two cameras and an angle (e.g., C, see FIG. 13A). In anotherexample implementation, both images are shifted toward the center of thefield of view (e.g., to the center of the left eye) based on thecorresponding camera position as relates to the position of the field ofview and until a portion (corresponding to the depth of view) of thecombined image is sharp.

FIG. 3A illustrates a top view perspective of a block diagram of animage capture configuration according to at least one exampleembodiment. As shown in FIG. 3A the configuration includes threeportions of an image 300. The three portions of the image 300 include afar portion 305 of the image 300, a mid portion 310 of the image 300 anda near portion 315 of the image 300. The far, mid and near indicatefocal depths associated with a viewer (e.g., having eyes 325-1 and325-2) of the image 300. In other words, the far portion 305 of theimage 300 is at a relatively far focal depth as viewed by the viewer,the mid portion 310 of the image 300 is relatively between the near andfar focal depth as viewed by the viewer, and the near portion 315 of theimage 300 is at a relatively near or close focal depth as viewed by theviewer.

FIG. 3B illustrates a front view perspective of the block diagram of theimage 300 capture configuration according to at least one exampleembodiment. As shown in FIG. 3B, the near portion 315 of the image 300is illustrated as in front of the mid portion 310 of the image 300.Further, the far portion 305 of the image 300 is illustrated as behind(and above for clarity) the near portion 315 of the image 300 and themid portion 310 of the image 300.

In an example embodiment, 3D image capture can include the use of morethan one camera each at a different position. This is illustrated bycameras 320-1, 320-2 and 320-3. Cameras 320-1, 320-2 and 320-3 can bedigital cameras configured to capture still images, a video and/orframes of a video.

FIG. 4 illustrates a block diagram of a method according to at least oneexample embodiment. As shown in FIG. 4, in step S405 a field of view isdetermined For example, the HMD 110 can include an accelerometer tomonitor a position or a movement of a user of the HMD's 110 head. Forexample, a device (e.g., a camera and/or an infrared sensor) external tothe HMD 110 can monitor a position or a movement of a user of the HMD's110 head. For example, the HMD 110 can include an eye tracking device(e.g., first eye tracking sensor 1220-L and second eye tracking sensor1220-R as shown in FIG. 12B) configured to determine a direction ofgaze. From the position and/or movement of the HMD 110 and/or thedirection of gaze, a field of view can be determined In other words, theposition and/or movement of the HMD 110 and/or the direction of gaze candetermine what (e.g., field of view) the user of the HMD 110 is lookingat.

In step S410 a depth of view is determined For example, the HMD 110 caninclude an eye tracking device (e.g., first eye tracking sensor 1220-Land second eye tracking sensor 1220-R) configured to determine the depthof view and or perform a measurement from which the depth of view can bedetermined. For example, the eye tracking device can measure a positionof the pupils of the user's eyes. The eye tracking device can measureconvergence of the pupils and determine angle C (see FIG. 13A). Angle Ccan be used to determine the depth of view. The depth processing module225 can be configured to use the measurements and/or an indication ofdepth of view received from the HMD 110 in order to determine the depthof view as it relates to an image.

In step S415 two or more right eye images and two or more left eyeimages are selected based on the field of view. For example, images canbe selected from a datastore (e.g., memory and/or database) and/or basedon a live capture. For example, the image selection module 220 canselect 2D images from the image/video source repository 215 based on anindication of a field of view for each of the left eye and the righteye. In an example implementation, a plurality of cameras (e.g., camera320-1, 320-2, 320-3 or 1230) can be used to capture images and/or video.The field of view for the left eye can be between two of the cameras(e.g., camera 320-1 and 320-2) and the field of view for the right eyecan be between two (e.g., different) cameras (e.g., camera 320-2 and320-3). Therefore, an image captured by each of the cameras flanking theleft eye (e.g., camera 320-1 and 320-2) can be selected and an imagecaptured by each of the cameras flanking the right eye (e.g., camera320-2 and 320-3) can be selected.

In step S420 a left eye image is generated based on the two or more lefteye images and the depth of view. For example, the left eye image can begenerated (e.g., by the image combining module 230) by combining the twoor more left eye images such that the combined image is sharp (e.g., notdistorted or not blurred) at the depth of view. In order to generate acombined image a processor (e.g., processor 210 executing a set ofinstructions) can shift a first image (e.g., as captured by camera320-1) with respect to a second image (e.g., as captured by camera320-2) captured at the same (or substantially the same) time as thefirst image until a portion of the combined image is sharp (e.g., theportion of the first image at the depth of view substantially overlapsthe portion of the second image at the same depth of view). In anexample implementation, the first image is shifted a number of pixelsbased on a distance between the two cameras and an angle (e.g., C, seeFIG. 13A). In another example implementation, both images are shiftedtoward the center of the field of view (e.g., to the center of the lefteye) until a portion (corresponding to the depth of view) of thecombined image is sharp.

Then the first image and the second image can be combined (e.g.,overlaid) with one another based on the shift. Further, a postprocessing may occur. For example, a color merging or adjustment may beperformed such that the combined portion of the image has substantiallythe same color palette as the portion of the image in the first imageand/or second image. For example, a color (or pixel data value) offsetmay be determined and applied across the combined image. The coloroffset can be a weighted offset based on a camera distance from thecenter of the field of view.

In step S425 a right eye image is generated based on the two or moreright eye images and the depth of view. For example, the right eye imagecan be generated (e.g., by the image combining module 230) by combiningthe two or more right eye images such that the combined image is sharp(e.g., not distorted or not blurred) at the depth of view. In order togenerate a combined image a processor (e.g., processor 210 executing aset of instructions) can shift a first image (e.g., as captured bycamera 320-2) with respect to a second image (e.g., as captured bycamera 320-3) captured at the same (or substantially the same) time asthe first image until a portion of the combined image is sharp (e.g.,the portion of the first image at the depth of view substantiallyoverlaps the portion of the second image at the same depth of view). Inan example implementation, the first image is shifted a number of pixelsbased on a distance between the two cameras and an angle (e.g., C, seeFIG. 13A). In another example implementation, both images are shiftedtoward the center of the field of view (e.g., to the center of the righteye) until a portion (corresponding to the depth of view) of thecombined image is sharp.

Then the first image and the second image can be combined (e.g.,overlaid) with one another based on the shift. Further, a postprocessing may occur. For example, a color merging or adjustment may beperformed such that the combined portion of the image has substantiallythe same color palette as the portion of the image in the first imageand/or second image. For example, a color (or pixel data value) offsetmay be determined and applied across the combined image. The coloroffset can be a weighted offset based on a camera distance from thecenter of the field of view.

In step S430 the left eye image and the right eye image are displayed onat least one display of the HMD 110. For example, the HMD 110 can have afirst display associated with the left eye and a second displayassociated with the right eye. The generated left eye image can berendered on the first display and the generated right eye image can berendered on the second display. In an example implementation, the HMD110 can include a single display including a first portion associatedwith the left eye and a second portion associated with the right eye.

FIGS. 5A, 5B and 5C illustrate image 300 as captured by camera 320-1,320-2 and 320-3, respectively. FIGS. 6A and 6B illustrate image 300 asviewed by eyes 325-1 and 325-2, respectively. As shown in FIG. 5B,camera 320-2 can capture image 300 substantially straight on as thereare minimal (if any) differences when comparing image 300 as shown inFIG. 3B to image 300 as shown in FIG. 5B. Contrast, image 300 ascaptured by camera 320-1 as shown in FIG. 5A. As can be seen, image 300is shown with the near portion 315 shifted to the right (or away fromcamera 320-1) as compared to the mid portion 310 and the far portion 305shifted to the left (or toward camera 320-1) as compared to the midportion 310. Further, image 300 as captured by camera 320-3 as shown inFIG. 5C is shown with the near portion 315 shifted to the left (or awayfrom camera 320-3) as compared to the mid portion 310 and the farportion 305 shifted to the right (or toward camera 320-3) as compared tothe mid portion 310.

As shown in FIG. 6A image 300 as viewed by eye 325-1 (e.g., as viewedwith eye 325-2 closed) is somewhat similar to image 300 as captured bycamera 320-1 or as illustrated in FIG. 5A. Accordingly, in FIG. 6A,image 300 is shown with the near portion 315 shifted to the right (oraway from eye 325-1) as compared to the mid portion 310 and the farportion 305 shifted to the left (or toward eye 325-1) as compared to themid portion 310. As shown in FIG. 6B image 300 as viewed by eye 325-2(e.g., as viewed with eye 325-1 closed) is somewhat similar to image 300as captured by camera 320-3 or as illustrated in FIG. 5C. Accordingly,in FIG. 6A, image 300 is shown with the near portion 315 shifted to theleft (or away from eye 325-2) as compared to the mid portion 310 and thefar portion 305 shifted to the right (or toward eye 325-2) as comparedto the mid portion 310.

FIGS. 7A, 7B and 7C illustrate the scene represented by image 300 asvisualized by the human visual system. For example, FIGS. 7A, 7B and 7Cillustrate image 300 as visualized by the human visual system whenviewed in real time or in the real world. FIG. 7A illustrates the scenerepresented by image 300 as visualized by the human visual system whenthe depth of focus is far. In other words, FIG. 7A illustrates avisualization of the scene represented by image 300 when viewed in realtime or in the real world and the eyes (e.g., eyes 205-1, 205-2 or325-1, 325-2) are focused on the scene represented by the far portion305 of the scene represented by the image 300.

As shown in FIG. 7A, the far portion 305 of the image 300 is as shown inFIG. 3B. In other words, the far portion 305 of the image 300 isillustrated as behind (and above for clarity) the near portion 315 ofthe image 300 and the mid portion 310 of the image 300. The far portion305 of the image 300 is also the shape (e.g., a single shaded star) asshown in FIG. 3B. However, in FIG. 7A, the near portion 315 of the image300 and the mid portion 310 of the image 300 are shown as distorted whencompared to the near portion 315 of the image 300 and the mid portion310 of the image 300 as shown in FIG. 3B. This is so, because the humanvisual system can only see clearly at one depth of focus.

As shown in FIG. 7A, the near portion 315 of the image 300 and the midportion 310 of the image 300 are shown as double vision portions. Forexample, the near portion 315 of the image 300 is split into threevisualized near portions 315-1, 315-2 and 315-3. Visualized near portion315-2 is shown as having the same shape and shading as the near portion315 of the image 300 shown in FIG. 3B. However, visualized near portion315-2 is shown as being narrower than the near portion 315 of the image300 shown in FIG. 3B. Visualized near portions 315-1 and 315-3 are shownas having the same shape and a lighter shading compared to the nearportion 315 of the image 300 shown in FIG. 3B. Visualized near portions315-1 and 315-3 are shown as extending the width of the near portion 315of the image 300 past the width of the mid portion 310 of the image 300similar to the mid portion 310 of the image 300 as shown in FIG. 3B.Visualized near portions 315-1 and 315-3 can represent the double visionportion of the near portion 315 of the image 300 when viewed in realtime or in the real world and the eyes (e.g., eyes 205-1, 205-2 or325-1, 325-2) are focused on the scene represented by the far portion305 of the image 300.

For example, the mid portion 310 of the image 300 is split into threevisualized mid portions 310-1, 310-2 and 310-3. Visualized mid portion310-2 is shown as having the same shape and shading as the mid portion310 of the image 300 shown in FIG. 3B. However, visualized mid portion310-2 is shown as being narrower than the mid portion 310 of the image300 shown in FIG. 3B. Visualized mid portions 310-1 and 310-3 are shownas having the same shape and a lighter shading compared to the midportion 310 of the image 300 shown in FIG. 3B. Visualized mid portions310-1 and 310-3 are shown as extending the width of the mid portion 310of the image 300 such that the width of the mid portion 310 of the image300 is wider than the mid portion 310 of the image 300 as shown in FIG.3B. Visualized mid portions 310-1 and 310-3 can represent the doublevision portion of the mid portion 310 of the image 300 when viewed inreal time or in the real world and the eyes (e.g., eyes 205-1, 205-2 or325-1, 325-2) are focused on the scene represented by the far portion305 of the image 300.

FIG. 7B illustrates the scene represented by image 300 as visualized bythe human visual system when the depth of focus is mid. In other words,FIG. 7B illustrates a visualization of the scene represented by image300 when viewed in real time or in the real world and the eyes (e.g.,eyes 205-1, 205-2 or 325-1, 325-2) are focused on the scene representedby the mid portion 310 of the scene represented by the image 300.

As shown in FIG. 7B, the mid portion 310 of the image 300 is as shown inFIG. 3B. In other words, the mid portion 310 of the image 300 isillustrated as behind the near portion 315 of the image 300 and in frontof the far portion 305 of the image 300. The mid portion 310 of theimage 300 shown in FIG. 7B is also the shape (e.g., a cross shadedrectangle) and size as shown in FIG. 3B. However, in FIG. 7B, the nearportion 315 of the image 300 and the far portion 305 of the image 300are shown as distorted when compared to the near portion 315 of theimage 300 and the far portion 305 of the image 300 as shown in FIG. 3B.This is so, because the human visual system can only see clearly at onedepth of focus.

As shown in FIG. 7B, the near portion 315 of the image 300 and the farportion 305 of the image 300 are shown as double vision portions. Forexample, the near portion 315 of the image 300 is split into threevisualized near portions 315-1, 315-2 and 315-3. Visualized near portion315-2 is shown as having the same shape and shading as the near portion315 of the image 300 shown in FIG. 3B. However, visualized near portion315-2 is shown as being narrower than the near portion 315 of the image300 shown in FIG. 3B. Visualized near portions 315-1 and 315-3 are shownas having the same shape and a lighter shading compared to the nearportion 315 of the image 300 shown in FIG. 3B. Visualized near portions315-1 and 315-3 are shown as extending the width of the near portion 315of the image 300 past the width of the mid portion 310 of the image 300similar to the mid portion 310 of the image 300 as shown in FIG. 3B.Visualized near portions 315-1 and 315-3 can represent the double visionportion of the near portion 315 of the image 300 when viewed in realtime or in the real world and the eyes (e.g., eyes 205-1, 205-2 or325-1, 325-2) are focused on the scene represented by the mid portion310 of the image 300.

For example, the far portion 305 of the image 300 is split into twovisualized far portions 305-1 and 305-2. Visualized far portions 305-1and 305-2 shown in FIG. 7B are shown as having the same shape (e.g., astar) and a lighter shading compared to the far portion 305 of the image300 shown in FIG. 3B. Visualized far portions 305-1 and 305-2 canrepresent the double vision portion of the far portion 305 of the image300 when viewed in real time or in the real world and the eyes (e.g.,eyes 205-1, 205-2 or 325-1, 325-2) are focused on the scene representedby the mid portion 310 of the image 300.

FIG. 7C illustrates the scene represented by image 300 as visualized bythe human visual system when the depth of focus is near. In other words,FIG. 7C illustrates a visualization of the scene represented by image300 when viewed in real time or in the real world and the eyes (e.g.,eyes 205-1, 205-2 or 325-1, 325-2) are focused on the scene representedby the near portion 315 of the scene represented by the image 300.

As shown in FIG. 7C, the near portion 315 of the image 300 is as shownin FIG. 3B. In other words, the near portion 315 of the image 300 isillustrated as in front of the mid portion 310 of the image 300 and infront of the far portion 305 of the image 300. The near portion 315 ofthe image 300 shown in FIG. 7C is also the shape (e.g., a dot shadedrectangle) and size as shown in FIG. 3B. However, in FIG. 7C, the midportion 310 of the image 300 and the far portion 305 of the image 300are shown as distorted when compared to the mid portion 310 of the image300 and the far portion 305 of the image 300 as shown in FIG. 3B. Thisis so, because the human visual system can only see clearly at one depthof focus.

As shown in FIG. 7C, the mid portion 310 of the image 300 and the farportion 305 of the image 300 are shown as double vision portions. Forexample, the mid portion 310 of the image 300 is split into threevisualized mid portions 310-1, 310-2 and 310-3. Visualized mid portion310-2 is shown as having the same shape and shading as the mid portion310 of the image 300 shown in FIG. 3B. However, visualized mid portion310-2 is shown as being narrower than the mid portion 310 of the image300 shown in FIG. 3B. Visualized mid portions 310-1 and 310-3 are shownas having the same shape and a lighter shading compared to the midportion 310 of the image 300 shown in FIG. 3B. Visualized mid portions310-1 and 310-3 are shown as extending the width of the mid portion 310of the image 300 such that the width of the mid portion 310 of the image300 is wider than the mid portion 310 of the image 300 as shown in FIG.3B. Visualized mid portions 310-1 and 310-3 can represent the doublevision portion of the mid portion 310 of the image 300 when viewed inreal time or in the real world and the eyes (e.g., eyes 205-1, 205-2 or325-1, 325-2) are focused on the scene represented by the near portion315 of the image 300.

For example, the far portion 305 of the image 300 is split into twovisualized far portions 305-1 and 305-2. Visualized far portions 305-1and 305-2 shown in FIG. 7C are shown as having the same shape (e.g., astar) and a lighter shading compared to the far portion 305 of the image300 shown in FIG. 3B. Visualized far portions 305-1 and 305-2 canrepresent the double vision portion of the far portion 305 of the image300 when viewed in real time or in the real world and the eyes (e.g.,eyes 205-1, 205-2 or 325-1, 325-2) are focused on the scene representedby the near portion 315 of the image 300.

The examples illustrated in FIGS. 7A, 7B and 7C can also be compared toeach other. For example, the aforementioned double vision portions areindicated as having a larger distortion the further the depth of focusis from the visualization. For example, FIG. 7B shows the image 300 whenfocused on the scene represented by the mid portion 310 of the image 300and FIG. 7C shows the image 300 when focused on the scene represented bythe near portion 315 of the image 300. Accordingly, the depth of focusas compared to the scene represented by the far portion 305 of the image300 is shown as further away in FIG. 7C than FIG. 7B. As such, thevisualized far portions 305-1 and 305-2 shown in FIG. 7C and FIG. 7B areshown in different positions. For example, the visualized far portions305-1 and 305-2 shown in FIG. 7B are closer together (e.g., lessdistorted) and substantially within the width of the mid portion 310 ascompared to the visualized far portions 305-1 and 305-2 shown in FIG.7C. A similar analysis can be done for the when the focus is on thescene represented by the mid portion 310 and the far portion 305 of theimage 300.

FIGS. 8A and 8B illustrate diagrams of combined images according to atleast one example embodiment. FIG. 8A illustrates a combined image 805based on combining image 300 as captured by camera 320-1 and 320-2. Thecombined image 805 is shown as combined at a focus depth based on thefar portion 305 of the image 300. In other words, the combined image 805is a combination of the image 300 from two different fields of view. Asshown in FIG. 8A, the far portion 305 of the combined image 805 issubstantially similar to the far portion 305 of the image 300 as shownin FIG. 3B. In other words, in FIG. 8A, the far portion 305 of thecombined image 805 is illustrated as behind (and above for clarity) thenear portion 315 and the mid portion 310. The far portion 305 of thecombined image 805 is also the shape (e.g., a single shaded star) asshown in FIG. 3B. However, in FIG. 8A, the far portion 305 of thecombined image 805 is shown shifted to the left as compared to the farportion 305 of the image 300 as shown in FIG. 3B. Alternatively, thenear portion 315 and the mid portion 310 are shown as shifted to theright. Further, in FIG. 8A, the near portion 315 and the mid portion 310are shown as distorted when compared to the near portion 315 of theimage 300 and the mid portion 310 of the image 300 as shown in FIG. 3B.

As shown in FIG. 8A, the near portion 315 and the mid portion 310 areshown as double vision portions. For example, the near portion 315 issplit into three near portions 315-1, 315-2 and 315-3. The near portion315-2 of the combined image 805 is shown as having the same shape andshading (e.g., is not distorted) as the near portion 315 of the image300 shown in FIG. 3B. However, the near portion 315-2 of the combinedimage 805 is shown as being narrower than the near portion 315 of theimage 300 shown in FIG. 3B. The near portions 315-1 and 315-3 of thecombined image 805 are shown as having the same shape and a lightershading compared to the near portion 315 of the image 300 shown in FIG.3B. The near portions 315-1 and 315-3 of the combined image 805 areshown as extending the width of the near portion 315 of the image 300past the width of the mid portion 310 of the image 300 similar to themid portion 310 of the image 300 as shown in FIG. 3B. The near portions315-1 and 315-3 of the combined image 805 are shown as shifted to theright as compared to the near portion 315 of the image 300 shown in FIG.3B.

For example, the mid portion 310 is split into three mid portions 310-1,310-2 and 310-3. The mid portion 310-2 of the combined image 805 isshown as having the same shape and shading as the mid portion 310 of theimage 300 shown in FIG. 3B. However, the mid portion 310-2 of thecombined image 805 is shown as being narrower than the mid portion 310of the image 300 shown in FIG. 3B. The mid portions 310-1 and 310-3 ofthe combined image 805 are shown as having the same shape and a lightershading compared to the mid portion 310 of the image 300 shown in FIG.3B. The mid portions 310-1 and 310-3 of the combined image 805 are shownas extending the width of the mid portion 310 of the image 300 such thatthe width of the mid portion 310 of the image 300 is wider than the midportion 310 of the image 300 as shown in FIG. 3B. The mid portions 310-1and 310-3 of the combined image 805 are shown as shifted to the right ascompared to the near portion 315 of the image 300 shown in FIG. 3B witha shift that is slightly less than the shift of the near portions 315-1and 315-3 of the combined image 805 (e.g., the near portion 315-2 of thecombined image 805 extends past the right boundary of mid portion 310-2,which is different than that which is shown in FIG. 7A).

FIG. 8B illustrates a combined image 810 based on combining image 300 ascaptured by camera 320-2 and 320-3. The combined image 810 is shown ascombined at a focus depth based on the far portion 305 of the image 300.In other words, the combined image 810 is a combination of the image 300from two different fields of view. As shown in FIG. 8B, the far portion305 of the combined image 810 is substantially similar to the farportion 305 of the image 300 as shown in FIG. 3B. In other words, inFIG. 8B, the far portion 305 of the combined image 810 is illustrated asbehind (and above for clarity) the near portion 315 and the mid portion310. The far portion 305 of the combined image 810 is also the shape(e.g., a single shaded star) as shown in FIG. 3B. However, in FIG. 8B,the far portion 305 of the combined image 805 is shown shifted to theright as compared to the far portion 305 of the image 300 as shown inFIG. 3B. Alternatively, the near portion 315 and the mid portion 310 areshown as shifted to the left. Further, in FIG. 8B, the near portion 315and the mid portion 310 are shown as distorted when compared to the nearportion 315 of the image 300 and the mid portion 310 of the image 300 asshown in FIG. 3B.

As shown in FIG. 8B, the near portion 315 and the mid portion 310 areshown as double vision portions. For example, the near portion 315 issplit into three near portions 315-1, 315-2 and 315-3. The near portion315-2 of the combined image 810 is shown as having the same shape andshading (e.g., is not distorted) as the near portion 315 of the image300 shown in FIG. 3B. However, the near portion 315-2 of the combinedimage 810 is shown as being narrower than the near portion 315 of theimage 300 shown in FIG. 3B. The near portions 315-1 and 315-3 of thecombined image 810 are shown as having the same shape and a lightershading compared to the near portion 315 of the image 300 shown in FIG.3B. The near portions 315-1 and 315-3 of the combined image 810 areshown as extending the width of the near portion 315 of the image 300past the width of the mid portion 310 of the image 300 similar to themid portion 310 of the image 300 as shown in FIG. 3B. The near portions315-1 and 315-3 of the combined image 810 are shown as shifted to theleft as compared to the near portion 315 of the image 300 shown in FIG.3B.

For example, the mid portion 310 is split into three mid portions 310-1,310-2 and 310-3. The mid portion 310-2 of the combined image 810 isshown as having the same shape and shading as the mid portion 310 of theimage 300 shown in FIG. 3B. However, the mid portion 310-2 of thecombined image 810 is shown as being narrower than the mid portion 310of the image 300 shown in FIG. 3B. The mid portions 310-1 and 310-3 ofthe combined image 810 are shown as having the same shape and a lightershading compared to the mid portion 310 of the image 300 shown in FIG.3B. The mid portions 310-1 and 310-3 of the combined image 810 are shownas extending the width of the mid portion 310 of the image 300 such thatthe width of the mid portion 310 of the image 300 is wider than the midportion 310 of the image 300 as shown in FIG. 3B. The mid portions 310-1and 310-3 of the combined image 810 are shown as shifted to the left ascompared to the near portion 315 of the image 300 shown in FIG. 3B witha shift that is slightly less than the shift of the near portions 315-1and 315-3 of the combined image 810 (e.g., the near portion 315-2 of thecombined image 810 extends past the left boundary of mid portion 310-1,which is different than that which is shown in FIG. 7A).

In order to generate a combined image (e.g., combined image 805 and/orcombined image 810) a processor (e.g., processor 210 executing a set ofinstructions) can shift a first image (e.g., as captured by camera320-1) with respect to a second image (e.g., as captured by camera320-2) captured at the same (or substantially the same) time as thefirst image until a portion of a combined image (e.g., the far portion305) is sharp (e.g., the portion of the first image substantiallyoverlaps the portion of the second image). In an example implementation,the first image is shifted a number of pixels based on a distancebetween the two cameras and an angle (e.g., C, see FIG. 13A).Alternatively, or in addition to, the processor can match a position ofa portion of an image (e.g., the far portion 305) in a first image(e.g., as captured by camera 320-1) with a position of a same portion ofan image (e.g., the far portion 305) in a second image (e.g., ascaptured by camera 320-2) captured at the same (or substantially thesame) time as the first image. Then the first image can be shifted basedon the matched position.

Then the first image and the second image can be combined (e.g.,overlaid) with one another based on the shift. Further, a postprocessing may occur. For example, a color merging or adjustment may beperformed such that the combined portion of the image has substantiallythe same color palette as the portion of the image in the first imageand/or second image. For example, a color (or pixel data value) offsetmay be determined and applied across the combined image.

FIG. 8C illustrates a visualized image 815 when displayed (and viewed)on a display of a HMD according to at least one example embodiment. Forexample, combined image 805 could be displayed on display 105-L andcombined image 810 could be displayed on display 105-R each of HMD 150as shown in FIG. 1B. The visualized image 815 can be the image asperceived by the human visual system as visualized in the full binocularoverlap visual field 1320 and/or the high resolution region of binocularoverlapping visual field 1325 (see FIG. 13A). In other words, FIG. 8Cillustrates a visualization of the scene represented by image 300 whenrendered on and viewed through a HMD using combined images 805 and 810.Further, by using the combined images 805 and 810 a perceived 3D imageis rendered without interpolating 3D images from captured 2D images.

As shown in FIG. 8C, the far portion 305 of the visualized image 815 issubstantially similar to the far portion 305 of the image 300 as shownin FIG. 3B. In other words, the far portion 305 of the visualized image815 is illustrated as behind (and above for clarity) the near portion315 of the visualized image 815 and the mid portion 310 of thevisualized image 815. The far portion 305 of the visualized image 815 isalso the shape (e.g., a single shaded star) as the far portion 305 ofthe image 300 as shown in FIG. 3B. However, in FIG. 8C, the near portion315 of the visualized image 815 and the mid portion 310 of thevisualized image 815 are shown as distorted when compared to the nearportion 315 of the image 300 and the mid portion 310 of the image 300 asshown in FIG. 3B. This is so, because the human visual system can onlysee clearly at one depth of focus.

As shown in FIG. 8C, the near portion 315 of the visualized image 815and the mid portion 310 of the visualized image 815 are shown as triplevision portions. For example, the near portion 315 of the visualizedimage 815 is split into five visualized near portions 315-1, 315-2,315-3, 315-4 and 315-5. Visualized near portion 315-2 is shown as havingthe same shape and shading as the near portion 315 of the image 300shown in FIG. 3B. However, visualized near portion 315-2 is shown asbeing narrower than the near portion 315 of the image 300 shown in FIG.3B. Visualized near portions 315-1 and 315-3 are shown as having thesame shape and a lighter shading compared to the near portion 315 of theimage 300 shown in FIG. 3B. Visualized near portions 315-4 and 315-5 areshown as having the same shape and a lighter shading compared tovisualized near portions 315-1 and 315-3. Visualized near portions315-1, 315-3, 315-4 and 315-5 are shown as extending the width of thenear portion 315-2 of the visualized image 815 past the width of the midportion 310 of the visualized image 815 similar to the mid portion 310of the image 300 as shown in FIG. 3B. Visualized near portions 315-1,315-3, 315-4 and 315-5 can represent the triple vision portion of thenear portion 315 of the image 300 when viewed through a HMD usingcombined images 805 and 810 and the eyes (e.g., eyes 205-1, 205-2 or325-1, 325-2) are focused on the far portion 305 of the image 300.

For example, the mid portion 310 of the visualized image 815 is splitinto five visualized mid portions 310-1, 310-2 a, 310-2 b, 310-3 and310-4. Visualized mid portions 310-2 a and 310-2 b of the visualizedimage 815 are shown as having the same shape and shading as the midportion 310 of the image 300 shown in FIG. 3B. However, visualized midportions 310-2 a and 310-2 b are shown as having a distortive portion310-4 between them, whereas the mid portion 310 of the image 300 shownin FIG. 3B is continuous. Visualized mid portions 310-1 and 310-3 of thevisualized image 815 are shown as having the same shape and a lightershading compared to the mid portion 310 of the image 300 shown in FIG.3B. Visualized mid portions 310-1 and 310-3 of the visualized image 815are shown as extending the width of the mid portion 310 of the image 300such that the width of the mid portion 310 of the image 300 is widerthan the mid portion 310 of the image 300 as shown in FIG. 3B.Visualized mid portions 310-1, 310-3 and 310-5 of the visualized image815 can represent the triple vision portion of the mid portion 310 ofthe image 300 when viewed through a HMD using combined images 805 and810 and the eyes (e.g., eyes 205-1, 205-2 or 325-1, 325-2) are focusedon the far portion 305 of the image 300.

FIGS. 9A and 9B illustrate diagrams of combined images according to atleast one example embodiment. FIG. 9A illustrates a combined image 905based on combining image 300 as captured by camera 320-1 and 320-2. Thecombined image 905 is shown as combined at a focus depth based on themid portion 310 of the image 300. In other words, the combined image 905is a combination of the image 300 from two different fields of view. Asshown in FIG. 9A, the mid portion 310 of the combined image 905 issubstantially similar to the mid portion 310 of the image 300 as shownin FIG. 3B. In other words, in FIG. 9A, the mid portion 310 of thecombined image 905 is illustrated as behind the near portion 315 and infront of the far portion 305. The mid portion 310 of the combined image905 is also the shape (e.g., a cross shaded rectangle) and size as shownin FIG. 3B. However, in FIG. 9A, the mid portion 310 of the combinedimage 905 is shown shifted to the left as compared to the mid portion310 of the image 300 as shown in FIG. 3B. Alternatively, the nearportion 315 and the far portion 305 are shown as shifted to the right.Further, in FIG. 9A, the near portion 315 and the far portion 305 areshown as distorted when compared to the near portion 315 of the image300 and the far portion 305 of the image 300 as shown in FIG. 3B.

As shown in FIG. 9A, the near portion 315 and the far portion 305 areshown as double vision portions. For example, the near portion 315 issplit into three near portions 315-1, 315-2 and 315-3. The near portion315-2 of the combined image 905 is shown as having the same shape andshading (e.g., is not distorted) as the near portion 315 of the image300 shown in FIG. 3B. However, the near portion 315-2 of the combinedimage 905 is shown as being narrower than the near portion 315 of theimage 300 shown in FIG. 3B. The near portions 315-1 and 315-3 of thecombined image 905 are shown as having the same shape and a lightershading compared to the near portion 315 of the image 300 shown in FIG.3B. The near portions 315-1 and 315-3 of the combined image 905 areshown as extending the width of the near portion 315 of the image 300past the width of the mid portion 310 of the image 300 similar to themid portion 310 of the image 300 as shown in FIG. 3B. The near portions315-1 and 315-3 of the combined image 905 are shown as shifted to theright as compared to the near portion 315 of the image 300 shown in FIG.3B.

For example, the far portion 305 is split into two far portions 305-1and 305-2. The far portions 305-1 and 305-2 shown in FIG. 9A are shownas having the same shape (e.g., a star) and a lighter shading comparedto the far portion 305 of the image 300 shown in FIG. 3B. The farportions 305-1 and 305-2 of the combined image 905 are shown as shiftedto the right as compared to the near portion 315 of the image 300 shownin FIG. 3B.

FIG. 9B illustrates a combined image 910 based on combining image 300 ascaptured by camera 320-2 and 320-3. The combined image 910 is shown ascombined at a focus depth based on the mid portion 310 of the image 300.In other words, the combined image 910 is a combination of the image 300from two different fields of view. As shown in FIG. 9B, the mid portion310 of the combined image 910 is substantially similar to the midportion 310 of the image 300 as shown in FIG. 3B. In other words, inFIG. 9B, the mid portion 310 of the combined image 910 is illustrated asbehind the near portion 315 and in front of the far portion 305. The farportion 305 of the combined image 910 is also the shape (e.g., a crossshaded rectangle) and size as shown in FIG. 3B. However, in FIG. 9B, themid portion 310 of the combined image 910 is shown shifted to the rightas compared to the mid portion 310 of the image 300 as shown in FIG. 3B.Alternatively, the near portion 315 and the far portion 305 are shown asshifted to the left. Further, in FIG. 9B, the near portion 315 and thefar portion 305 are shown as distorted when compared to the near portion315 of the image 300 and the far portion 305 of the image 300 as shownin FIG. 3B.

As shown in FIG. 9B, the near portion 315 and the far portion 305 areshown as double vision portions. For example, the near portion 315 issplit into three near portions 315-1, 315-2 and 315-3. The near portion315-2 of the combined image 910 is shown as having the same shape andshading (e.g., is not distorted) as the near portion 315 of the image300 shown in FIG. 3B. However, the near portion 315-2 of the combinedimage 910 is shown as being narrower than the near portion 315 of theimage 300 shown in FIG. 3B. The near portions 315-1 and 315-3 of thecombined image 910 are shown as having the same shape and a lightershading compared to the near portion 315 of the image 300 shown in FIG.3B. The near portions 315-1 and 315-3 of the combined image 910 areshown as extending the width of the near portion 315 of the image 300past the width of the mid portion 310 of the image 300 similar to themid portion 310 of the image 300 as shown in FIG. 3B. The near portions315-1 and 315-3 of the combined image 910 are shown as shifted to theleft as compared to the near portion 315 of the image 300 shown in FIG.3B.

For example, the far portion 305 is split into two far portions 305-1and 305-2. The far portions 305-1 and 305-2 shown in FIG. 9B are shownas having the same shape (e.g., a star) and a lighter shading comparedto the far portion 305 of the image 300 shown in FIG. 3B. The farportions 305-1 and 305-2 of the combined image 910 are shown as shiftedto the left as compared to the near portion 315 of the image 300 shownin FIG. 3B.

In order to generate a combined image (e.g., combined image 905 and/orcombined image 910) a processor (e.g., processor 210 executing a set ofinstructions) can shift a first image (e.g., as captured by camera320-1) with respect to a second image (e.g., as captured by camera320-2) captured at the same (or substantially the same) time as thefirst image until a portion of a combined image (e.g., the mid portion310) is sharp (e.g., the portion of the first image substantiallyoverlaps the portion of the second image). In an example implementation,the first image is shifted a number of pixels based on a distancebetween the two cameras and an angle (e.g., C, see FIG. 2A).Alternatively, or in addition to, the processor can match a position ofa portion of an image (e.g., the mid portion 310) in a first image(e.g., as captured by camera 320-1) with a position of a same portion ofan image (e.g., the far portion 305) in a second image (e.g., ascaptured by camera 320-2) captured at the same (or substantially thesame) time as the first image. Then the first image can be shifted basedon the matched position.

Then the first image and the second image can be combined (e.g.,overlaid) with one another based on the shift. Further, a postprocessing may occur. For example, a color merging or adjustment may beperformed such that the combined portion of the image has substantiallythe same color palette as the portion of the image in the first imageand/or second image. For example, a color (or pixel data value) offsetmay be determined and applied across the combined image.

FIG. 9C illustrates a visualized image 915 when displayed (and viewed)on a display of a HMD according to at least one example embodiment. Forexample, combined image 905 could be displayed on display 105-L andcombined image 910 could be displayed on display 105-R each of HMD 150as shown in FIG. 1B. The visualized image 915 can be the image asperceived by the human visual system as visualized in the full binocularoverlap visual field 1320 and/or the high resolution region of binocularoverlapping visual field 1325 (see FIG. 13A). In other words, FIG. 9Cillustrates a visualization of the scene represented by image 300 whenrendered on and viewed through a HMD using combined images 905 and 910.Further, by using the combined images 905 and 910 a perceived 3D imageis rendered without interpolating 3D images from captured 2D images.

As shown in FIG. 9C, the mid portion 310 of the visualized image 915 issubstantially similar to the mid portion 310 of the image 300 as shownin FIG. 3B. In other words, in FIG. 9C, the mid portion 310 of thevisualized image 915 is illustrated as behind the near portion 315 andin front of the far portion 305. The mid portion 310 of the visualizedimage 915 is also the shape (e.g., a cross shaded rectangle) and size asshown in FIG. 3B. However, in FIG. 9C, the near portion 315 of thevisualized image 915 and the far portion 305 of the visualized image 915are shown as distorted when compared to the near portion 315 of theimage 300 and the far portion 305 of the image 300 as shown in FIG. 3B.This is so, because the human visual system can only see clearly at onedepth of focus.

As shown in FIG. 9C, the near portion 315 of the visualized image 915and the far portion 305 of the visualized image 915 are shown as triplevision portions. For example, the near portion 315 of the visualizedimage 915 is split into five visualized near portions 315-1, 315-2,315-3, 315-4 and 315-5. Visualized near portion 315-2 is shown as havingthe same shape and shading as the near portion 315 of the image 300shown in FIG. 3B. However, visualized near portion 315-2 is shown asbeing narrower than the near portion 315 of the image 300 shown in FIG.3B. Visualized near portions 315-1 and 315-3 are shown as having thesame shape and a lighter shading compared to the near portion 315 of theimage 300 shown in FIG. 3B. Visualized near portions 315-4 and 315-5 areshown as having the same shape and a lighter shading compared tovisualized near portions 315-1 and 315-3. Visualized near portions315-1, 315-3, 315-4 and 315-5 are shown as extending the width of thenear portion 315-2 of the visualized image 915 past the width of the midportion 310 of the visualized image 915 similar to the mid portion 310of the image 300 as shown in FIG. 3B. Visualized near portions 315-1,315-3, 315-4 and 315-5 can represent the triple vision portion of thenear portion 315 of the image 300 when viewed through a HMD usingcombined images 905 and 910 and the eyes (e.g., eyes 205-1, 205-2 or325-1, 325-2) are focused on the far portion 305 of the image 300.

For example, the far portion 305 of the visualized image 915 is splitinto three visualized far portions 305-1, 305-2 and 305-3. Visualizedfar portion 305-2 of the visualized image 915 is shown as having thesame shape and shading as the far portion 305 of the image 300 shown inFIG. 3B. However, visualized far portions 305-1 and 305-3 of thevisualized image 915 are shown as having the same shape and a lightershading compared to the far portion 305 of the image 300 shown in FIG.3B. Visualized far portions 305-1 and 305-3 of the visualized image 915can represent the triple vision portion of the far portion 305 of theimage 300 when viewed through a HMD using combined images 905 and 910and the eyes (e.g., eyes 205-1, 205-2 or 325-1, 325-2) are focused onthe far portion 305 of the image 300.

FIGS. 10A and 10B illustrate diagrams of combined images according to atleast one example embodiment. FIG. 10A illustrates a combined image 1005based on combining image 300 as captured by camera 320-1 and 320-2. Thecombined image 1005 is shown as combined at a focus depth based on thenear portion 315 of the image 300. In other words, the combined image1005 is a combination of the image 300 from two different fields ofview. As shown in FIG. 10A, the near portion 315 of the combined image1005 is substantially similar to the near portion 315 of the image 300as shown in FIG. 3B. In other words, in FIG. 10A, the mid portion 315 ofthe combined image 1005 is illustrated as in front of the mid portion310 and in front of the far portion 305. The near portion 315 of thecombined image 1005 is also the shape (e.g., a dot shaded rectangle) andsize as shown in FIG. 3B. However, in FIG. 10A, the near portion 315 ofthe combined image 1005 is shown shifted to the right as compared to thenear portion 315 of the image 300 as shown in FIG. 3B. Alternatively,the mid portion 310 and the far portion 305 are shown as shifted to theleft. Further, in FIG. 10A, the mid portion 310 and the far portion 305are shown as distorted when compared to the mid portion 310 of the image300 and the far portion 305 of the image 300 as shown in FIG. 3B.

As shown in FIG. 10A, the mid portion 310 and the far portion 305 areshown as double vision portions. For example, the mid portion 310 issplit into three mid portions 310-1, 310-2 and 310-3. The mid portion310-2 of the combined image 1005 is shown as having the same shape andshading as the mid portion 310 of the image 300 shown in FIG. 3B.However, the mid portion 310-2 of the combined image 1005 is shown asbeing narrower than the mid portion 310 of the image 300 shown in FIG.3B. The mid portions 310-1 and 310-3 of the combined image 1005 areshown as having the same shape and a lighter shading compared to the midportion 310 of the image 300 shown in FIG. 3B. The mid portions 310-1and 310-3 of the combined image 1005 are shown as extending the width ofthe mid portion 310 of the image 300 such that the width of the midportion 310 of the image 300 is wider than the mid portion 310 of theimage 300 as shown in FIG. 3B. The mid portions 310-1, 310-2 and 310-3of the combined image 1005 are shown as shifted to the left.

For example, the far portion 305 is split into two far portions 305-1and 305-2. The far portions 305-1 and 305-2 shown in FIG. 10A are shownas having the same shape (e.g., a star) and a lighter shading comparedto the far portion 305 of the image 300 shown in FIG. 3B. The farportions 305-1 and 305-2 of the combined image 1005 are shown as shiftedto the left as compared to the near portion 315 of the image 300 shownin FIG. 3B.

FIG. 10B illustrates a combined image 1010 based on combining image 300as captured by camera 320-2 and 320-3. The combined image 1010 is shownas combined at a focus depth based on the near portion 315 of the image300. In other words, the combined image 1010 is a combination of theimage 300 from two different fields of view. As shown in FIG. 10B, thenear portion 315 of the combined image 1010 is substantially similar tothe near portion 315 of the image 300 as shown in FIG. 3B. In otherwords, in FIG. 10B, the near portion 315 of the combined image 1010 isillustrated as in front of the mid portion 310 and in front of the farportion 305. The near portion 315 of the combined image 1010 is also theshape (e.g., a dot shaded rectangle) and size as shown in FIG. 3B.However, in FIG. lOB, the near portion 315 of the combined image 1010 isshown shifted to the left as compared to the near portion 315 of theimage 300 as shown in FIG. 3B. Alternatively, the mid portion 310 andthe far portion 305 are shown as shifted to the right. Further, in FIG.lOB, the mid portion 310 and the far portion 305 are shown as distortedwhen compared to the mid portion 310 of the image 300 and the farportion 305 of the image 300 as shown in FIG. 3B.

As shown in FIG. lOB, the mid portion 310 and the far portion 305 areshown as double vision portions. For example, the mid portion 310 issplit into three mid portions 310-1, 310-2 and 310-3. The mid portion310-2 of the combined image 1010 is shown as having the same shape andshading as the mid portion 310 of the image 300 shown in FIG. 3B.However, the mid portion 310-2 of the combined image 1010 is shown asbeing narrower than the mid portion 310 of the image 300 shown in FIG.3B. The mid portions 310-1 and 310-3 of the combined image 1010 areshown as having the same shape and a lighter shading compared to the midportion 310 of the image 300 shown in FIG. 3B. The mid portions 310-1and 310-3 of the combined image 1010 are shown as extending the width ofthe mid portion 310 of the image 300 such that the width of the midportion 310 of the image 300 is wider than the mid portion 310 of theimage 300 as shown in FIG. 3B. The mid portions 310-1, 310-2 and 310-3of the combined image 1005 are shown as shifted to the right.

For example, the far portion 305 is split into two far portions 305-1and 305-2. The far portions 305-1 and 305-2 shown in FIG. lOB are shownas having the same shape (e.g., a star) and a lighter shading comparedto the far portion 305 of the image 300 shown in FIG. 3B. The farportions 305-1 and 305-2 of the combined image 1010 are shown as shiftedto the right as compared to the near portion 315 of the image 300 shownin FIG. 3B.

In order to generate a combined image (e.g., combined image 1005 and/orcombined image 1010) a processor (e.g., processor 210 executing a set ofinstructions) can shift a first image (e.g., as captured by camera320-1) with respect to a second image (e.g., as captured by camera320-2) captured at the same (or substantially the same) time as thefirst image until a portion of a combined image (e.g., the near portion315) is sharp (e.g., the portion of the first image substantiallyoverlaps the portion of the second image). In an example implementation,the first image is shifted a number of pixels based on a distancebetween the two cameras and an angle (e.g., C, see FIG. 2A).Alternatively, or in addition to, the processor can match a position ofa portion of an image (e.g., the near portion 315) in a first image(e.g., as captured by camera 320-1) with a position of a same portion ofan image (e.g., the near portion 315) in a second image (e.g., ascaptured by camera 320-2) captured at the same (or substantially thesame) time as the first image. Then the first image can be shifted basedon the matched position.

Then the first image and the second image can be combined (e.g.,overlaid) with one another based on the shift. Further, a postprocessing may occur. For example, a color merging or adjustment may beperformed such that the combined portion of the image has substantiallythe same color palette as the portion of the image in the first imageand/or second image. For example, a color (or pixel data value) offsetmay be determined and applied across the combined image.

FIG. 10C illustrates a visualized image 1015 when displayed (and viewed)on a display of a HMD according to at least one example embodiment. Forexample, combined image 1005 could be displayed on display 105-L andcombined image 1010 could be displayed on display 105-R each of HMD 150as shown in FIG. 1B. The visualized image 1015 can be the image asperceived by the human visual system as visualized in the full binocularoverlap visual field 1320 and/or the high resolution region of binocularoverlapping visual field 1325 (see FIG. 13A). In other words, FIG. 10Cillustrates a visualization of the scene represented by image 300 whenrendered on and viewed through a HMD using combined images 1005 and1010. Further, by using the combined images 1005 and 1010 a perceived 3Dimage is rendered without interpolating 3D images from captured 2Dimages.

As shown in FIG. 10C, the mid portion 310 of the visualized image 1015is substantially similar to the near portion 315 of the image 300 asshown in FIG. 3B. In other words, in FIG. 10C, the near portion 315 ofthe visualized image 1015 is illustrated as in front of the mid portion310 and in front of the far portion 305. The near portion 315 of thevisualized image 1015 is also the shape (e.g., a dot shaded rectangle)and size as shown in FIG. 3B. However, in FIG. 10C, the mid portion 310of the visualized image 1015 and the far portion 305 of the visualizedimage 915 are shown as distorted when compared to the near portion 315of the image 300 and the far portion 305 of the image 300 as shown inFIG. 3B. This is so, because the human visual system can only seeclearly at one depth of focus.

As shown in FIG. 10C, the mid portion 310 of the visualized image 1015and the far portion 305 of the visualized image 1015 are shown as triplevision portions. For example, the mid portion 310 of the visualizedimage 1015 is split into five visualized mid portions 310-1, 310-2,310-3, 310-4 and 310-5. Visualized mid portion 310-2 is shown as havingthe same shape and shading as the mid portion 310 of the image 300 shownin FIG. 3B. However, visualized mid portion 310-2 is shown as beingnarrower than the mid portion 310 of the image 300 shown in FIG. 3B.Visualized mid portions 310-1 and 310-3 are shown as having the sameshape and a lighter shading compared to the near portion 315 of theimage 300 shown in FIG. 3B. Visualized mid portions 310-4 and 310-5 areshown as having the same shape and a lighter shading compared tovisualized mid portions 310-1 and 310-3. Visualized mid portions 310-1,310-3, 310-4 and 310-5 are shown as extending the width of the midportion 310-2 of the visualized image 915 past the width of the midportion 310 of the visualized image 915 similar to the mid portion 310of the image 300 as shown in FIG. 3B. Visualized mid portions 310-1,310-3, 310-4 and 310-5 can represent the triple vision portion of thenear portion 315 of the image 300 when viewed through a HMD usingcombined images 1005 and 1010 and the eyes (e.g., eyes 205-1, 205-2 or325-1, 325-2) are focused on the near portion 315 of the image 300.

For example, the far portion 305 of the visualized image 1015 is splitinto three visualized far portions 305-1, 305-2 and 305-3. Visualizedfar portion 305-2 of the visualized image 1015 is shown as having thesame shape and shading as the far portion 305 of the image 300 shown inFIG. 3B. However, visualized far portions 305-1 and 305-3 of thevisualized image 1015 are shown as having the same shape and a lightershading compared to the far portion 305 of the image 300 shown in FIG.3B. Visualized far portions 305-1 and 305-3 of the visualized image 915can represent the triple vision portion of the far portion 305 of theimage 300 when viewed through a HMD using combined images 1005 and 1010and the eyes (e.g., eyes 205-1, 205-2 or 325-1, 325-2) are focused onthe near portion 315 of the image 300.

FIGS. 11A and 11B illustrate diagrams of a side by side comparison ofvisualized images at a far depth of view according to at least oneexample embodiment. The visualized image 300 illustrated in FIG. 11A isthe same visualized image 300 illustrated in FIG. 7A. Accordingly, thevisualized image 300 illustrated in FIG. 11A is described above withregard to FIG. 7A. The visualized image 815 illustrated in FIG. 11B isthe same visualized image 815 illustrated in FIG. 8C. Accordingly, thevisualized image 815 illustrated in FIG. 11B is described above withregard to FIG. 8C. The visualized image 300 illustrated in FIG. 11A cancorrespond to what the human visual system would actually see whenlooking at the scene corresponding to image 300 shown in FIG. 3B. Thevisualized image 815 illustrated in FIG. 11B can correspond to what thehuman visual system would actually see when viewing a 3D imagecorresponding to image 300 shown in FIG. 3B rendered on an HMD usingcombined images 805 and 810 according to an example implementation.

Comparing the visualized image 300 illustrated in FIG. 11A with thevisualized image 815 illustrated in FIG. 11B, the portion of thevisualized images at the focal depth is substantially the same. In otherwords, the far portion 305 of the image 300 shown in FIG. 11A issubstantially the same as the far portion 305 of the image 815 shown inFIG. 11B. The mid portion 310 and the near portion 315 of the image 300shown in FIG. 11A are different than the mid portion 310 and the nearportion 315 of the image 815 shown in FIG. 11B. For example, the midportion 310 and the near portion 315 of the image 300 shown in FIG. 11Aare doubled and the mid portion 310 and the near portion 315 of theimage 815 shown in FIG. 11B are tripled. The difference between thevisualized image 300 shown in FIG. 11A and the rendered visualized image815 shown in FIG. 11B is acceptable, because the human visual system canonly see clearly at one depth of focus.

FIGS. 11C and 11D illustrate diagrams of a side by side comparison ofvisualized images at a mid depth of view according to at least oneexample embodiment. The visualized image 300 illustrated in FIG. 11C isthe same visualized image 300 illustrated in FIG. 7B. Accordingly, thevisualized image 300 illustrated in FIG. 11C is described above withregard to FIG. 7B. The visualized image 915 illustrated in FIG. 11D isthe same visualized image 915 illustrated in FIG. 9C. Accordingly, thevisualized image 915 illustrated in FIG. 11D is described above withregard to FIG. 9C. The visualized image 300 illustrated in FIG. 11C cancorrespond to what the human visual system would actually see whenlooking at the scene corresponding to image 300 shown in FIG. 3B. Thevisualized image 915 illustrated in FIG. 11D can correspond to what thehuman visual system would actually see when viewing a 3D imagecorresponding to image 300 shown in FIG. 3B rendered on an HMD usingcombined images 905 and 910 according to an example implementation.

Comparing the visualized image 300 illustrated in FIG. 11C with thevisualized image 915 illustrated in FIG. 11D, the portion of thevisualized images at the focal depth is substantially the same. In otherwords, the mid portion 310 of the image 300 shown in FIG. 11C issubstantially the same as the mid portion 310 of the image 915 shown inFIG. 11D. The far portion 305 and the near portion 315 of the image 300shown in FIG. 11C are different than the far portion 305 and the nearportion 315 of the image 915 shown in FIG. 11D. For example, the farportion 305 and the near portion 315 of the image 300 shown in FIG. 11Care doubled and the far portion 305 and the near portion 315 of theimage 915 shown in FIG. 11D are tripled. The difference between thevisualized image 300 shown in FIG. 11C and the rendered visualized image915 shown in FIG. 11D is acceptable, because the human visual system canonly see clearly at one depth of focus.

FIGS. 11E and 11F illustrate diagrams of a side by side comparison ofvisualized images at a near depth of view according to at least oneexample embodiment. The visualized image 300 illustrated in FIG. 11E isthe same visualized image 300 illustrated in FIG. 7C. Accordingly, thevisualized image 300 illustrated in FIG. 11E is described above withregard to FIG. 7C. The visualized image 1015 illustrated in FIG. 11F isthe same visualized image 1015 illustrated in FIG. 10C. Accordingly, thevisualized image 1015 illustrated in FIG. 11F is described above withregard to FIG. 10C. The visualized image 300 illustrated in FIG. 11E cancorrespond to what the human visual system would actually see whenlooking at the scene corresponding to image 300 shown in FIG. 3B. Thevisualized image 1015 illustrated in FIG. 11F can correspond to what thehuman visual system would actually see when viewing a 3D imagecorresponding to image 300 shown in FIG. 3B rendered on an HMD usingcombined images 1005 and 1010 according to an example implementation.

Comparing the visualized image 300 illustrated in FIG. 11E with thevisualized image 1015 illustrated in FIG. 11F, the portion of thevisualized images at the focal depth is substantially the same. In otherwords, the near portion 315 of the image 300 shown in FIG. 11E issubstantially the same as the near portion 315 of the image 1015 shownin FIG. 11F. The far portion 305 and the mid portion 310 of the image300 shown in FIG. 11E are different than the far portion 305 and the midportion 310 of the image 1015 shown in FIG. 11F. For example, the farportion 305 and the mid portion 310 of the image 300 shown in FIG. 11Eare doubled and the far portion 305 and the mid portion 310 of the image1015 shown in FIG. 11F are tripled. The difference between thevisualized image 300 shown in FIG. 11E and the rendered visualized image1015 shown in FIG. 11F is acceptable, because the human visual systemcan only see clearly at one depth of focus.

FIG. 12A illustrates a block diagram of a head mounted display (HMD)according to at least one example embodiment. As shown in FIG. 12A, theHMD 1200 includes a display 1205, a first portion of the display 1210and a second portion of the display 1215. The first portion of thedisplay 1210 can be a portion of the display 1205 on which an eye isfocused or the field of view of a user of the HMD 1200. The secondportion of the display 1215 can be a portion of the display 1205peripheral to the focus of the eye or the periphery view of a user ofthe HMD 1200.

According to example embodiments, a 2D image and/or 2D frame of a videocan be chosen for display on each display of a HMD based on tracking afield of view and depth of view of an eye using an eye tracking sensorin order to render 3D images and/or 3D video. As a result, the user ofthe HMD has a 3D experience without interpolating 3D images fromcaptured 2D images.

FIG. 12B illustrates a block diagram of a head mounted display (HMD)according to at least one example embodiment. As shown in FIG. 12B, theHMD 1250 includes a first display 1205-L, a second display 1205-R, afirst eye tracking sensor 1220-L and a second eye tracking sensor1220-R. The first display 1205-L can include a first portion of thedisplay 1210-L and a second portion of the display 1215-L. The firstportion of the display 1210-L can be a portion of the first display1205-L on which an eye is focused or the field of view of a user of theHMD 1250. The second portion of the display 1215-L can be a portion ofthe first display 1205-L peripheral to the focus of the eye or theperiphery view of a user of the HMD 1250. The second display 1205-R caninclude a first portion of the display 1210-R and a second portion ofthe display 1215-R. The first portion of the display 1210-R can be aportion of the second display 1205-R on which an eye is focused or thefield of view of a user of the HMD 1250. The second portion of thedisplay 1215-R can be a portion of the second display 1205-R peripheralto the focus of the eye or the periphery view of a user of the HMD 1250.

The first portion of the display 1210-L of the first display 1205-L andthe first portion of the display 1210-R of the second display 1205-R canbe configured to display images and/or video data in a field of visionassociated with a high resolution region of binocular overlap (describedin more detail below). The second portion of the display 1215-L of thefirst display 1205-L and the second portion of the display 1215-R of thesecond display 1205-R may be configured to display images and/or videoin a field of vision outside of, or peripheral to, the high resolutionregion of binocular overlap, including a further region of binocularoverlap including a lower resolution. The first display 1205-L can beassociated with a left eye and can be configured to display a left eyeimage in a 3D image or video. The second display 1205-R can beassociated with a right eye and can be configured to display a right eyeimage in the 3D image or video. In an alternative embodiment, the firstdisplay 1205-L and the second display 1205-R are formed from oneintegral display panel capable of showing an image that is partitionedinto two parts comprising left and right images.

The first eye tracking sensor 1220-L and the second eye tracking sensor1220-R can be configured to track a position of a corresponding eye. Thefirst eye tracking sensor 1220-L and the second eye tracking sensor1220-R can be a camera, an infrared sensor, a combination thereof and/orthe like. In an example implementation, the first eye tracking sensor1220-L and the second eye tracking sensor 1220-R can be configured toproject a low powered infrared light into a user's eye. Then, thereflected infrared light can be detected and a measurement of thereflected infrared light (e.g., as a Perkinje image) can be used todetermine an eye's position. In another example implementation, adirection of gaze can be determined by locating the pupil relative tothe rest of the eye under ambient light conditions. The first eyetracking sensor 1220-L and the second eye tracking sensor 1220-R canmeasure the absolute direction of the user's gaze relative to a point onthe HMD and/or an angular displacement of the pupil upon each eyemovement.

In an example implementation, the first eye tracking sensor 1220-L andthe second eye tracking sensor 1220-R can be used to determine adirection of gaze (also referred to as view perspective in thisdisclosure) and a depth of gaze of a user of the HMD. The direction ofgaze and the depth of gaze can be used to determine an image to bedisplayed on each of the first display 1205-L and the second display1205-R.

FIG. 13A illustrates a schematic representation from a top view ofhorizontal visual fields. As shown in FIG. 13A, eyes 1305-1, 1305-2(e.g., human eyes), including pupils 1330-1, 1330-2, can visuallyperceive a left visual field 1310 and a right visual field 1315. Withinthe left visual field 1310 and the right visual field 1315, the eyes1305-1, 1305-2 can visually perceive a full binocular overlap visualfield 1320 which may be as large as 120 deg. A sub-region of fullbinocular overlap can be referred to as the high resolution region ofbinocular overlapping visual field 1325. As shown in FIG. 13B (andthroughout this description), a vertical plane which hereinafter themidsagittal frontal plane bisects the head between the eyes, and avertical plane that hereinafter the vertical frontal plane intersectsthe head orthogonal to the midsagittal plane at a position that bisectsthe eyes 1305-1 and 1305-2. FIG. 13C shows a horizontal plane thatextends in a direction left and right (or horizontally) with respect tothe eyes 1305-1, 1305-2 and that also bisects the eyes. The plane inFIG. 13C may be called the horizontal frontal plane. The three planesdefined in FIG. 13B and 13C can intersect at the midpoint of a linesegment extending from the center of the left eye to the center of theright eye.

The fovea is the central portion of the retina of each of the eyes1305-1, 1305-2 that perceives the highest resolution. The direction ofgaze (illustrated by vector G parallel to the midsagittal plane) may bedefined by a vector from the center of the fovea through the center ofthe pupil. Neither eye 1305-1 nor eye 1305-2 turns or rotatessufficiently to allow the direction of gaze to scan the full horizontalvisual field 1310 or 1315. Therefore, imagery beyond the turning limitof the eyes 1305-1, 1305-2 may not be viewed by the fovea (although suchimagery will be viewed by other parts of the retina).

It should be noted that although the fovea subtends only a small arc,the rotation of the eyes can extend the range of angles over which adisplay should match foveal resolution. When the user's eyes move andthe direction of gaze changes, resolution matching the fovea isdesirable over the range of comfortable gaze scanning. The range ofcomfortable gaze scanning is approximately 15 degrees in any directionwith respect to vector G in FIG. 13A. The gaze can scan over largerangles with progressively more discomfort as the scan angle increasesbeyond 15 degrees from vector G.

In an example implementation of an HMD (while referring to FIG. 13C),all horizontal angles (e.g., angles along the horizontal frontal planesuch as θ) can be measured (e.g., using the first eye tracking sensor1220-L and the second eye tracking sensor 1220-R) with respect to thesagittal frontal plane (e.g., the plane of head symmetry centeredbetween the eyes). The left visual field 1310 and the right visual field1315 regions represent the visual fields of the left and right eyes thatcan be supplied images with partial binocular overlap that can be asmuch as 120 degrees per eye (e.g., the full binocular overlap visualfield 1320 region, matching the overlap in the human visual system). Thehigh resolution region of binocular overlap visual field 1325 can besupplied by left and right displays (e.g., display 105-L and display105-R).

As such, the eyes 1305-1, 1305-2 can visualize a 3D image based onselecting images projected onto (or displayed by) left and rightdisplays representing the visual fields of the left and right eyes inthe high resolution region of binocular overlap visual field 1325 (andto some extent the full binocular overlap visual field 1320 region). Inother words, selecting images for display in the full binocular overlapvisual field 1320 region can match the overlap in the human visualsystem resulting in the human visual system visualize a 3D image and/or3D video. As discussed in more detail below, the depth of gaze can beused to more realistically represent the human visual system.

According to an example implementation, an image for projection on adisplay for each eye can be based on direction of gaze shown in relationto point A and angle θ shown in FIG. 13C. Each image can then bemodified by a depth of vision based on angle C shown in FIG. 13A. Forexample, point A, angle 0 and angle C can be determined based onmeasurements captured using the first eye tracking sensor 1220-L and thesecond eye tracking sensor 1220-R. Point A, angle θ and/or angle C canbe used to determine the field of view and/or the depth of viewdiscussed above. For example, angle C can indicate a depth of view andpoint A in combination with angle θ can indicate a field of view.

FIG. 14 shows an example of a computer device 1400 and a mobile computerdevice 1450, which may be used with the techniques described here.Computing device 1400 is intended to represent various forms of digitalcomputers, such as laptops, desktops, workstations, personal digitalassistants, servers, blade servers, mainframes, and other appropriatecomputers. Computing device 1450 is intended to represent various formsof mobile devices, such as personal digital assistants, cellulartelephones, smart phones, and other similar computing devices. Thecomponents shown here, their connections and relationships, and theirfunctions, are meant to be exemplary only, and are not meant to limitimplementations of the inventions described and/or claimed in thisdocument.

Computing device 1400 includes a processor 1402, memory 1404, a storagedevice 1406, a high-speed interface 1408 connecting to memory 1404 andhigh-speed expansion ports 1410, and a low speed interface 1412connecting to low speed bus 1414 and storage device 1406. Each of thecomponents 1402, 1404, 1406, 1408, 1410, and 1412, are interconnectedusing various busses, and may be mounted on a common motherboard or inother manners as appropriate. The processor 1402 can processinstructions for execution within the computing device 1400, includinginstructions stored in the memory 1404 or on the storage device 1406 todisplay graphical information for a GUI on an external input/outputdevice, such as display 1416 coupled to high speed interface 1408. Inother implementations, multiple processors and/or multiple buses may beused, as appropriate, along with multiple memories and types of memory.Also, multiple computing devices 1400 may be connected, with each deviceproviding portions of the necessary operations (e.g., as a server bank,a group of blade servers, or a multi-processor system).

The memory 1404 stores information within the computing device 1400. Inone implementation, the memory 1404 is a volatile memory unit or units.In another implementation, the memory 1404 is a non-volatile memory unitor units. The memory 1404 may also be another form of computer-readablemedium, such as a magnetic or optical disk.

The storage device 1406 is capable of providing mass storage for thecomputing device 1400. In one implementation, the storage device 1406may be or contain a computer-readable medium, such as a floppy diskdevice, a hard disk device, an optical disk device, or a tape device, aflash memory or other similar solid state memory device, or an array ofdevices, including devices in a storage area network or otherconfigurations. A computer program product can be tangibly embodied inan information carrier. The computer program product may also containinstructions that, when executed, perform one or more methods, such asthose described above. The information carrier is a computer- ormachine-readable medium, such as the memory 1404, the storage device1406, or memory on processor 1402.

The high speed controller 1408 manages bandwidth-intensive operationsfor the computing device 1400, while the low speed controller 1412manages lower bandwidth-intensive operations. Such allocation offunctions is exemplary only. In one implementation, the high-speedcontroller 1408 is coupled to memory 1404, display 1416 (e.g., through agraphics processor or accelerator), and to high-speed expansion ports1410, which may accept various expansion cards (not shown). In theimplementation, low-speed controller 1412 is coupled to storage device1406 and low-speed expansion port 1414. The low-speed expansion port,which may include various communication ports (e.g., USB, Bluetooth,Ethernet, wireless Ethernet) may be coupled to one or more input/outputdevices, such as a keyboard, a pointing device, a scanner, or anetworking device such as a switch or router, e.g., through a networkadapter.

The computing device 1400 may be implemented in a number of differentforms, as shown in the figure. For example, it may be implemented as astandard server 1420, or multiple times in a group of such servers. Itmay also be implemented as part of a rack server system 1424. Inaddition, it may be implemented in a personal computer such as a laptopcomputer 1422. Alternatively, components from computing device 1400 maybe combined with other components in a mobile device (not shown), suchas device 1450. Each of such devices may contain one or more ofcomputing device 1400, 1450, and an entire system may be made up ofmultiple computing devices 1400, 1450 communicating with each other.

Computing device 1450 includes a processor 1452, memory 1464, aninput/output device such as a display 1454, a communication interface1466, and a transceiver 1468, among other components. The device 1450may also be provided with a storage device, such as a microdrive orother device, to provide additional storage. Each of the components1450, 1452, 1464, 1454, 1466, and 1468, are interconnected using variousbuses, and several of the components may be mounted on a commonmotherboard or in other manners as appropriate.

The processor 1452 can execute instructions within the computing device1450, including instructions stored in the memory 1464. The processormay be implemented as a chipset of chips that include separate andmultiple analog and digital processors. The processor may provide, forexample, for coordination of the other components of the device 1450,such as control of user interfaces, applications run by device 1450, andwireless communication by device 1450.

Processor 1452 may communicate with a user through control interface1458 and display interface 1456 coupled to a display 1454. The display1454 may be, for example, a TFT LCD (Thin-Film-Transistor Liquid CrystalDisplay) or an OLED (Organic Light Emitting Diode) display, or otherappropriate display technology. The display interface 1456 may compriseappropriate circuitry for driving the display 1454 to present graphicaland other information to a user. The control interface 1458 may receivecommands from a user and convert them for submission to the processor1452. In addition, an external interface 1462 may be provide incommunication with processor 1452, to enable near area communication ofdevice 1450 with other devices. External interface 1462 may provide, forexample, for wired communication in some implementations, or forwireless communication in other implementations, and multiple interfacesmay also be used.

The memory 1464 stores information within the computing device 1450. Thememory 1464 can be implemented as one or more of a computer-readablemedium or media, a volatile memory unit or units, or a non-volatilememory unit or units. Expansion memory 1474 may also be provided andconnected to device 1450 through expansion interface 1472, which mayinclude, for example, a SIMM (Single In Line Memory Module) cardinterface. Such expansion memory 1474 may provide extra storage spacefor device 1450, or may also store applications or other information fordevice 1450. Specifically, expansion memory 1474 may includeinstructions to carry out or supplement the processes described above,and may include secure information also. Thus, for example, expansionmemory 1474 may be provide as a security module for device 1450, and maybe programmed with instructions that permit secure use of device 1450.In addition, secure applications may be provided via the SIMM cards,along with additional information, such as placing identifyinginformation on the SIMM card in a non-hackable manner.

The memory may include, for example, flash memory and/or NVRAM memory,as discussed below. In one implementation, a computer program product istangibly embodied in an information carrier. The computer programproduct contains instructions that, when executed, perform one or moremethods, such as those described above. The information carrier is acomputer- or machine-readable medium, such as the memory 1464, expansionmemory 1474, or memory on processor 1452, that may be received, forexample, over transceiver 1468 or external interface 1462.

Device 1450 may communicate wirelessly through communication interface1466, which may include digital signal processing circuitry wherenecessary. Communication interface 1466 may provide for communicationsunder various modes or protocols, such as GSM voice calls, SMS, EMS, orMMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS, among others.Such communication may occur, for example, through radio-frequencytransceiver 1468. In addition, short-range communication may occur, suchas using a Bluetooth, Wi-Fi, or other such transceiver (not shown). Inaddition, GPS (Global Positioning System) receiver module 1470 mayprovide additional navigation- and location-related wireless data todevice 1450, which may be used as appropriate by applications running ondevice 1450.

Device 1450 may also communicate audibly using audio codec 1460, whichmay receive spoken information from a user and convert it to usabledigital information. Audio codec 1460 may likewise generate audiblesound for a user, such as through a speaker, e.g., in a handset ofdevice 1450. Such sound may include sound from voice telephone calls,may include recorded sound (e.g., voice messages, music files, etc.) andmay also include sound generated by applications operating on device1450.

The computing device 1450 may be implemented in a number of differentforms, as shown in the figure. For example, it may be implemented as acellular telephone 1480. It may also be implemented as part of a smartphone 1482, personal digital assistant, or other similar mobile device.

Various implementations of the systems and techniques described here canbe realized in digital electronic circuitry, integrated circuitry,specially designed ASICs (application specific integrated circuits),computer hardware, firmware, software, and/or combinations thereof Thesevarious implementations can include implementation in one or morecomputer programs that are executable and/or interpretable on aprogrammable system including at least one programmable processor, whichmay be special or general purpose, coupled to receive data andinstructions from, and to transmit data and instructions to, a storagesystem, at least one input device, and at least one output device.Various implementations of the systems and techniques described here canbe realized as and/or generally be referred to herein as a circuit, amodule, a block, or a system that can combine software and hardwareaspects. For example, a module may include the functions/acts/computerprogram instructions executing on a processor (e.g., a processor formedon a silicon substrate, a GaAs substrate, and the like) or some otherprogrammable data processing apparatus.

Some of the above example embodiments are described as processes ormethods depicted as flowcharts. Although the flowcharts describe theoperations as sequential processes, many of the operations may beperformed in parallel, concurrently or simultaneously. In addition, theorder of operations may be re-arranged. The processes may be terminatedwhen their operations are completed, but may also have additional stepsnot included in the figure. The processes may correspond to methods,functions, procedures, subroutines, subprograms, etc.

Methods discussed above, some of which are illustrated by the flowcharts, may be implemented by hardware, software, firmware, middleware,microcode, hardware description languages, or any combination thereofWhen implemented in software, firmware, middleware or microcode, theprogram code or code segments to perform the necessary tasks may bestored in a machine or computer readable medium such as a storagemedium. A processor(s) may perform the necessary tasks.

Specific structural and functional details disclosed herein are merelyrepresentative for purposes of describing example embodiments. Exampleembodiments, however, be embodied in many alternate forms and should notbe construed as limited to only the embodiments set forth herein.

It will be understood that, although the terms first, second, etc. maybe used herein to describe various elements, these elements should notbe limited by these terms. These terms are only used to distinguish oneelement from another. For example, a first element could be termed asecond element, and, similarly, a second element could be termed a firstelement, without departing from the scope of example embodiments. Asused herein, the term and/or includes any and all combinations of one ormore of the associated listed items.

It will be understood that when an element is referred to as beingconnected or coupled to another element, it can be directly connected orcoupled to the other element or intervening elements may be present. Incontrast, when an element is referred to as being directly connected ordirectly coupled to another element, there are no intervening elementspresent. Other words used to describe the relationship between elementsshould be interpreted in a like fashion (e.g., between versus directlybetween, adjacent versus directly adjacent, etc.).

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of exampleembodiments. As used herein, the singular forms a, an and the areintended to include the plural forms as well, unless the context clearlyindicates otherwise. It will be further understood that the termscomprises, comprising, includes and/or including, when used herein,specify the presence of stated features, integers, steps, operations,elements and/or components, but do not preclude the presence or additionof one or more other features, integers, steps, operations, elements,components and/or groups thereof

It should also be noted that in some alternative implementations, thefunctions/acts noted may occur out of the order noted in the figures.For example, two figures shown in succession may in fact be executedconcurrently or may sometimes be executed in the reverse order,depending upon the functionality/acts involved.

Unless otherwise defined, all terms (including technical and scientificterms) used herein have the same meaning as commonly understood by oneof ordinary skill in the art to which example embodiments belong. Itwill be further understood that terms, e.g., those defined in commonlyused dictionaries, should be interpreted as having a meaning that isconsistent with their meaning in the context of the relevant art andwill not be interpreted in an idealized or overly formal sense unlessexpressly so defined herein.

Portions of the above example embodiments and corresponding detaileddescription are presented in terms of software, or algorithms andsymbolic representations of operation on data bits within a computermemory. These descriptions and representations are the ones by whichthose of ordinary skill in the art effectively convey the substance oftheir work to others of ordinary skill in the art. An algorithm, as theterm is used here, and as it is used generally, is conceived to be aself-consistent sequence of steps leading to a desired result. The stepsare those requiring physical manipulations of physical quantities.Usually, though not necessarily, these quantities take the form ofoptical, electrical, or magnetic signals capable of being stored,transferred, combined, compared, and otherwise manipulated. It hasproven convenient at times, principally for reasons of common usage, torefer to these signals as bits, values, elements, symbols, characters,terms, numbers, or the like.

In the above illustrative embodiments, reference to acts and symbolicrepresentations of operations (e.g., in the form of flowcharts) that maybe implemented as program modules or functional processes includeroutines, programs, objects, components, data structures, etc., thatperform particular tasks or implement particular abstract data types andmay be described and/or implemented using existing hardware at existingstructural elements. Such existing hardware may include one or moreCentral Processing Units (CPUs), digital signal processors (DSPs),application-specific-integrated-circuits, field programmable gate arrays(FPGAs) computers or the like.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise, or as is apparent from the discussion,terms such as processing or computing or calculating or determining ofdisplaying or the like, refer to the action and processes of a computersystem, or similar electronic computing device, that manipulates andtransforms data represented as physical, electronic quantities withinthe computer system's registers and memories into other data similarlyrepresented as physical quantities within the computer system memoriesor registers or other such information storage, transmission or displaydevices.

Note also that the software implemented aspects of the exampleembodiments are typically encoded on some form of non-transitory programstorage medium or implemented over some type of transmission medium. Theprogram storage medium may be magnetic (e.g., a floppy disk or a harddrive) or optical (e.g., a compact disk read only memory, or CD ROM),and may be read only or random access. Similarly, the transmissionmedium may be twisted wire pairs, coaxial cable, optical fiber, or someother suitable transmission medium known to the art. The exampleembodiments not limited by these aspects of any given implementation.

Lastly, it should also be noted that whilst the accompanying claims setout particular combinations of features described herein, the scope ofthe present disclosure is not limited to the particular combinationshereafter claimed, but instead extends to encompass any combination offeatures or embodiments herein disclosed irrespective of whether or notthat particular combination has been specifically enumerated in theaccompanying claims at this time.

What is claimed is:
 1. A method comprising: receiving an indication of afield of view associated with a three-dimensional (3D) image beingdisplayed on a head mount display (HMD); receiving an indication of adepth of view associated with the 3D image being displayed on the HMD;selecting a first right eye image and a second right eye image based onthe field of view; combining the first right eye image and the secondright eye image based on the depth of view; selecting a first left eyeimage and a second left eye image based on the field of view; andcombining the first left eye image and the second left eye image basedon the depth of view.
 2. The method of claim 1, wherein selecting of thefirst right eye image and of the second right eye image include:determining a right eye position of a user of the HMD; selecting thefirst right eye image as an image taken by a first camera positioned tothe right of the right eye position; and selecting the second right eyeimage as an image taken by a second camera positioned to the left of theright eye position.
 3. The method of claim 1, wherein selecting of thefirst left eye image and of the second left eye image include:determining a left eye position of a user of the HMD; selecting thefirst left eye image as an image taken by a first camera positioned tothe right of the left eye position; and selecting the second left eyeimage as an image taken by a second camera positioned to the left of theleft eye position.
 4. The method of claim 1, wherein the first right eyeimage, the second right eye image, the first left eye image and thesecond left eye image are selected from a plurality of images capturedat substantially the same moment in time.
 5. The method of claim 1,wherein the combining of the first right eye image and the second righteye image includes shifting the first right eye image with respect tothe second right eye image until a portion of a combined image, based onthe depth of view, is sharp.
 6. The method of claim 1, wherein thecombining of the first right eye image and the second right eye imageincludes shifting both the first right eye image and the second righteye image toward the center of the field of view until a portion of acombined image, based on the depth of view, is sharp.
 7. The method ofclaim 1, wherein the combining of the first right eye image and thesecond right eye image includes color merging such that a portion of thecombined image has substantially a same color palette as a correspondingportion of at least one of the first right eye image and the secondright eye image.
 8. The method of claim 1, wherein the combining of thefirst right eye image and the second right eye image includes colormerging using a color offset based on a weighted offset associated witha camera distance from the center of the field of view.
 9. The method ofclaim 1, wherein the first left eye image, the second left eye image,the first left eye image and the second left eye image are selected froma plurality of images captured at substantially the same moment in time.10. The method of claim 1, wherein the combining of the first left eyeimage and the second left eye image includes shifting the first left eyeimage with respect to the second left eye image until a portion of acombined image, based on the depth of view, is sharp.
 11. The method ofclaim 1, wherein the combining of the first left eye image and thesecond left eye image includes shifting both the first left eye imageand the second left eye image toward the center of the field of viewuntil a portion of a combined image, based on the depth of view, issharp.
 12. The method of claim 1, wherein the combining of the firstleft eye image and the second left eye image includes color merging thecombined image such that a portion of the combined image hassubstantially a same color palette as a corresponding portion of atleast one of the first left eye image and the second left eye image. 13.The method of claim 1, wherein the combining of the first left eye imageand the second left eye image includes color merging the combined imageusing a color offset based on a weighted offset associated with a cameradistance from the center of the field of view.
 14. A method comprising:streaming a three-dimensional (3D) video to a head mount display (HMD),each frame of the 3D video including a left eye image and a right eyeimage; and generating a subsequent frame includes: determining a fieldof view associated with the 3D video; determining a depth of viewassociated with the 3D video; selecting a first right eye image and asecond right eye image based on the field of view; combining the firstright eye image and the second right eye image as the right eye imagebased on the depth of view; selecting a first left eye image and asecond left eye image based on the field of view; and combining thefirst left eye image and the second left eye image as the left eye imagebased on the depth of view.
 15. The method of claim 14, whereinselecting of the first right eye image and of the second right eye imageincludes: determining a right eye position of a user of the HMD,selecting the first right eye image as an image taken by a first camerapositioned to the right of the right eye position, and selecting thesecond right eye image as an image taken by a second camera positionedto the left of the right eye position; and selecting of the first lefteye image and of the second left eye image include: determining a lefteye position of a user of the HMD, selecting the first left eye image asan image taken by a first camera positioned to the right of the left eyeposition, and selecting the second left eye image as an image taken by asecond camera positioned to the left of the left eye position.
 16. Themethod of claim 14, wherein the first right eye image, the second righteye image, the first left eye image and the second left eye image areselected from a plurality of images captured at substantially the samemoment in time.
 17. The method of claim 14, wherein the combining of thefirst right eye image and the second right eye image includes shiftingthe first right eye image with respect to the second right eye imageuntil a portion of a combined image, based on the depth of view, issharp, and the combining of the first left eye image and the second lefteye image includes shifting the first left eye image with respect to thesecond left eye image until a portion of a combined image, based on thedepth of view, is sharp.
 18. The method of claim 14, wherein thecombining of the first right eye image and the second right eye imageincludes shifting both the first right eye image and the second righteye image toward the center of the field of view until a portion of acombined image, based on the depth of view, is sharp, and the combiningof the first left eye image and the second left eye image includesshifting both the first left eye image and the second left eye imagetoward the center of the field of view until a portion of a combinedimage, based on the depth of view, is sharp.
 19. The method of claim 14,wherein the combining of the first left eye image and the second lefteye image includes color merging the combined image.
 20. Anon-transitory computer-readable storage medium having stored thereoncomputer executable program code which, when executed on a computersystem, causes the computer system to perform steps comprising:receiving an indication of a field of view associated with athree-dimensional (3D) image being displayed on a head mount display(HMD); receiving an indication of a depth of view associated with the 3Dimage being displayed on the HMD; selecting a first right eye image anda second right eye image based on the field of view; combining the firstright eye image and the second right eye image based on the depth ofview; selecting a first left eye image and a second left eye image basedon the field of view; and combining the first left eye image and thesecond left eye image based on the depth of view.